Continuous Distributed Counting for Non-monotonous Streams

We consider the continual count tracking problem in a distributed environment where the input is anaggregate stream originating from k distinct sites and the updates are allowed to be non-monotonous, i.e. both incre-ments and decrements are allowed. The goal is to continually track the count within a prescribed relative accuracyat the lowest possible communication cost. Specifically, we consider an adversarial setting where the input valuesare selected and assigned to sites by an adversary but the order is according to a random permutation or is a randomi.i.d process. The input stream of values is allowed to be non-monotonous with an unknown drift −1 ≤ μ ≤ 1 wherethe case μ = 1 corresponds to the special case of a monotonous stream of only non-negative updates. We show thata randomized algorithm guarantees to track the count accurately with high probability and has the expected commu-nication cost Õ(min{√k/(|μ| ),√kn/ , n}), for an input stream of length n, and establish matching lower bounds.This improves upon previously best known algorithm whose expected communication cost is Θ̃(min{√k/ , n}) thatapplies only to an important but more restrictive class of monotonous input streams, and our results are substantiallymore positive than the communication complexity of Ω(n) under fully adversarial input. We also show how our frame-work can also accommodate other types of random input streams, including fractional Brownian motion that has beenwidely used to model temporal long-range dependencies observed in many natural phenomena. Last but not least, weshow how our non-monotonous counter can be applied to track the second frequency moment and to a Bayesian linearregression problem.